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I. INTRODUCTION 

Control over physical systems exerted by an external agent is ubiquitous in physics and engineering, where it is 
used for purposes such as stabilizing unstable dynamics or increasing system performance. Studying the properties of 
controlled systems is a fundamental task for engineering purposes and from a fundamental physical point of view [l| . 

The operation of the controller modifies the system dynamics, and the system in turn can also influence the controller 
decisions. Controllers are called closed-loop or feedback controllers if their actions are dependent on the state of the 
system and called open-loop if otherwise. In other words, closed-loop or feedback controllers actuate the system using 
the information about the system state, while open-loop controllers actuate the system independently of the system 
state. 

It is intuitively clear that using information about the system state is potentially useful to improve the controller 
action on the system and increase the system performance (e.g., the power output or the efficiency). However, 
the ceiling of such performance enhancement from making use of system state information is not clearly quantified. 
In particular, a complete and general development is lacking for calculating the performance in feedback controlled 
systems using thermodynamic principles. The aim of this paper is to present and put in context several open questions 
in this field. 

In the following section we briefly review some selected results about the relation between information and feedback. 
Next, in Section IlIIl we present several of the still unsolved questions and discuss their implications. Finally, our 
general conclusions are presented in Section ITVl 



II. STATE-OF-THE-ART 

In this section, we introduce some helpful results in the context of the below-mcntioncd open problems. The results 
presented here will be useful for one to get insight into the open questions and address their answers. 

In Subsectio nJII Al we comment some of the main results found in the framework of Maxwell's demon and computa- 
tion theory |2|-l8| . In Subsection IIIBI we summarize recent results about the relation between the achievable decrease 
of entropy and the information about the system gathered by the controller @-[n|- In Subsection III CI we show recent 
results concerning performance and information in a relevant stochastic system, namely, a feedback flashing ratchet. 
In Subsection III Dl we summarize recent results on the thermodynamics of feedback controlled systems. Finally, in 
Subsection III El we comment several very recent results that generalize statistical physics identities and relations to 
feedback controlled systems. 



A. Maxwell's Demon 

The trade-off between information and feedback control has been a difficult task since the birth of Maxwell's 
demon a hypothetical being that gathers information about a system to decrease the entropy of it without 
performing work. Szilard's seminal work Q contains the basic ingredients of the interrelation between information 
theory and thermodynamics, which was later precisely stated in the Landauer's principle: The erasure of one bit 
of information at temperature T implies an energetic cost of at least ksT In 2 (4j. Bennett [j| pointed out that the 
Landauer's principle is essential to preserve the second law of thermodynamics in feedback systems, as the controller 
must erase its memory after each cycle to allow the whole system to truly operate cyclically. How to achieve the 
shorter description for the memory record of the demon in order to minimize the energetic erasure cost was established 
by Zurek [|, % by using an algorithmic complexity approach. He proposed to compress the memory record of the 
demon before the erasure. This compression of the memory can be done, in principle, without energy expenditure — 
i.e., reversibly. The reversibility of the compression process is related to the fact that a memory can be erased at zero 
energy cost provided its state is known, i.e., stored in another memory Q. This implies that the minimum energy 
required to erase the memory of the demon is not given by the number of bits stored, but by the minimum number 
of bits in which the information contained in the memory can be stored (after compression) . This important result 
raises the attention on the relevance of considering whether the information is redundant or not. 
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B. Information-Theoretic Limits of Control 

A fundamental magnitude in information theory [l2j is the Shannon entropy of a random variable X . If X takes 
values x with probability px(x), then the Shannon entropy is given, in nats (1 bit = ln2 nats), by 

H(X) = -J2Px(x)ln Px (x) (1) 

X 

and it is a measure of the average uncertainty of the random variable [HJ . (When X is the macrostate of a system 
and x is its microstate, H(X) is related with the statistical physics entropy S of the system by S = kgH). On the 
other hand, the conditional entropy of X given another random variable C is 

H(X\C) = J2pc(c)H(X\C = c) (2) 

c 

with H(X\C = c) = — ^2 x Px\c( x \ c ) ^ n Px\c( x \ c )- From these magnitudes, the mutual information between two 
random variables, which is the information that one variable contains about the other one, is computed as 

I(X;C) = H(X)-H(X\C) (3) 

Hence, the mutual information can be viewed as the reduction of uncertainty of one of the random variables due to 
the knowledge of the other [l2j . 

Fundamental limits on the controllability of dynamical systems can be studied in the light of information theory 0- 
ITlj . Recently, it has been established that the maximum amount of entropy AiJ c i ose( j that can be extracted from any 
dynamical system by a single closed-loop actuation is upper bounded by the maximum entropy decrease achievable 
by open-loop control A_ff opcn plus the mutual information between the controller C and the state of the system 

AH closcd < AH opcn + I{X- C) (4) 

This result has provided insight into the relations between information and feedback, explicitly quantifying and show- 
ing one of the limitations of attainable performance improvement with using certain amount of system information. 
The analogous results for the physical entropy of a physical system has been recently obtained for repeated closed- 
loop actuations, see Section III D I and [13| . Also recently the application of information theory to biochemical reaction 
networks has allowed to show the limits of the ability to control fluctuations in molecular abundances inside the cells 
as a function of the number of signaling events [l4| • 



C. Feedback Controlled Ratchets and Information 



Let us now briefly review some related results applied to Brownian ratchets, viz., stochastic systems that are capable 
of rectifying thermal fluctuations by working out of equilibrium. Recently, feedback controlled versions of Brownian 
ratchets have been studied [l5l - f3"o| and experimentally realized [3l[ . (See Figure [1] for a Brownian ratchet that is 
based on the flashing of a potential.) 

These feedback controlled ratchets can improve the performance over its open-loop counterparts thanks to the 
information the controller gets about the system state. These information-dependent ratchets are relevant in chemical 
and biological systems (33l435j | , and also in nanotechnology devices [l5l - [23L l3lj ] . Other ratchets and Brownian motors 
inspired by Maxwell's demon present a directed motion when there is a temperature difference between two different 
parts of the motor (one part could be thought as the controller and the other as the controlled system) [3(| l37j . 
However, for these later systems no systematic study on the role of information has been performed. On the other 
hand, for feedback flashing ratchets, relations between the information / (about the system) gathered by the controller 
per step and the performance have been recently obtained [H,[3^|. These relations have been established for a collective 
feedback flashing ratchet compounded of one or few particles, considering the optimal protocol for one particle and 
its generalization to few particles. In this protocol the controller gathers the sign of the net force over the particles 
if the potential would be on. If the sign is measured with a perfect measurement device and it is transmitted to the 
controller through a noiseless channel, the controller gets one bit of information about the system (the maximum 
value of the information / per step for this system). However, if the measurement or the transmission is imperfect, 
the controller gets less than one bit of information about the system, and this limits the increase of performance 
improvement that can be obtained thanks to the information. 
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FIG. 1. In a feedback flashing ratchets a controller with information of the state of the system rectifies the thermal fluctuations 
of Brownian particles by switching on and off a periodic potential (which can be asymmetric, as the one depicted in the plot, 
or symmetric). This rectification of thermal fluctuations generates particle flux even against an external force, thus performing 
work [3411,1311, HI]- 



Let us call v pen the maximum flux that can be achieved with an open-loop control protocol, and i> c iosed the 
maximum flux that can be obtained with a closed-loop control protocol that uses an amount of information / per 
measurement step. The computations in |38| have shown that the maximum increase in the flux that can be obtained 
has an upper bound proportional to the square root of the information / received in each step, i.e., 

^closed - Vopen < G\\fl (5) 

(with C\ being a constant depending on the system's characteristics). 

An analogous bound has been found for the maximum power output in [39| . Let us call P pen the maximum power 
output that can be obtained with an open-loop control, and -P c iosed the maximum power output that can be obtained 
with a closed-loop control that uses an amount of information / per measurement step. It has been shown in [39| 
that the maximum increase in the power output has an upper bound proportional to the information / gathered by 
the controller per step, i.e., 



-^closed — -Popcn < C%1 (6) 

(with C*2 being a constant depending on the system's characteristics). 

The previous bounds [Equations ([5]) and ©] provide insight into the limitations of performance enhancement that 
can be obtained with a given amount of information. In particular, they have already been used to assess the feasibility 
of an experimental realization of feedback ratchets before its implementation [22, O |3l| . In addition, these bounds 
are an example of the type of predictions that a general theory would provide about the relations between information 
and feedback control. 



D. Thermodynamics of Feedback Controlled Systems 



Correlated repeated actuation of the feedback controller is present when the controller repeatedly actuates the 
system without waiting for the relaxation of the system to equilibrium between actuations. (Thus, adiabatic approxi- 
mations [40| cannot be applied.) In the correlated repeated actuation, the information received by the controller from 
the different measurements has redundancies. These redundancies have to be consistently taken into account as they 
reduce the possibilities to increase the performance up to the bounds that would be predicted from the known results 
for uncorrelated repeated actuations 043- 

Recently, the entropy reduction of a system due to the correlated repeated actuation of an external controller has 
been computed This was the main lacking ingredient to establish the thermodynamics of feedback controlled 

systems (i.e., to be able to compute the thermodynamic potentials and its relations for these systems). In the following 
paragraphs we include an excerpt of 13[ summarizing the main results. 

A control step of a controller involves: measuring the system, deciding the control action to take given the measure- 
ment output, and acting on the system following the control action chosen. From the point of view adopted in (ljjj . 
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i.e., the point of view of the system, we are just concerned about which action the controller has taken on the system, 
whose consequence is to modify the evolution of the system. We are not concerned about the state of the controller. 

Let us call X k := X(t k ) the macrostate of a general physical system at the kth control step of the controller (at 
time tfc), and C k the action taken by the controller (it is not a specification of the state of the controller). The entropy 
just before the first measurement is given by the statistical entropy 

S l = PX 1 W lnPx i W ='■ k B H ( X l) (?) 

with X the set of possible microstates of the system. If the measurement implies the control action C\ = c, the 
entropy of the system decreases to 

k B H{X x \C x = c) := -k B J^Px^cM^Px^cMc) (8) 

because knowing that the control action is C\ — c gives additional information about the actual microstate of the 
system, due to the correlation between measurements by the controller and control actions. Therefore, the average 
entropy after the first control step can be obtained by the following weighted average over the set C of all possible 
control actions: 

S ? = J2 Pc ^ kBH ( X ^ Cl = c ) =: k sH{X x \C x ) (9) 

cGC 

Hence the average variation of the entropy at the first step is 

ASi = S? - S b x = k B H{X x \C x ) - k B H(X x ) =: -k B I(X x ;C x ) (10) 

i.e., it is k B times the (minus) mutual information (l2| between X x and C x given in nats (In 2 nats = 1 bit). 

After the first measurement we have a set of macrostates (one for each possible control action). Each of the 
macrostates evolves to give an entropy k B H(X2\C\ = c) just before the second control step. The average entropy of 
the system after the second step is 

S 2 = E Pc 2Cl (c',c)k B H(X 2 \C 2 = c',C\ =c) = fc B fr(X a |C7 2j Ci) (11) 

c.c'ec 

and the average variation of the entropy at this second control step is 

A5 2 = 5" - S b 2 = fcflfrpklCa.d) - k B H{X 2 \C x ) = -k B I(X 2 ; C 2 \C X ) (12) 

This conditioning of the mutual information shows that the entropy of the system is only reduced by new non- 
redundant information. Analogously we get for the average entropy reduction of the fc-th step 

AS fe = -k B I{X k] C k \C k . u . . . ,Ci) = -k B H(C k \C k ^,. . . , d) + k B H(C k \C k ^,. . . , C u X k ) (13) 

Therefore, the total average entropy reduction due to the information used in M control steps is 
AS'info = Sfcli ASfc, i.e., 

M 

AS info = -k B 1(C k ;X k \C k -i, . . . , Ci) (14) 

k=l 

or equivalently 

M 

AS info = -k B H(C M , + H{C k \C k -x, Ci,X k ) (15) 

k=l 

The first term in this last equation gives the maximum entropy decrease attainable, and the second term gives the 
reduction on this maximum due to decorrelations between the control actions and the state of the system (decor- 
relations that can be due to errors and noises affecting the controller operation). These results have a broad range 
of applications including deterministic and non-deterministic feedback controls, and they indicate that the average 




FIG. 2. Markovian particle pump with two lattice sites between barriers. The particle is in contact with a thermal bath at 
temperature T, and an external controller measures the particle location and raises the closest barrier to the left. The result 
is that the particle is further and further constrained to the right, increasing its potential energy. This is a simple feedback 
controlled system that extracts useful work from the entropy reduction due to the information about the system used by the 
external feedback controller 



entropy reduction due to the information used can be computed in terms of the joint probabilities for the state of the 
system and the control actions history. 

These results have also allowed us to establish the thermodynamics of feedback controlled systems. As an applica- 
tion, we have computed the efficiency of isothermal feedback controlled system, and as an example, we have applied 
the results to compute the efficiency of a Markovian particle pump. This isothermal feedback controlled system can 
extract work from a single heat bath, thanks to the use of information, see Figure [5] and [l3| . For an experimental 
demonstration of a Markovian particle pump, see [4l|. Very recently other motors that extract work from a single 
heat bath have been studied |42h45|. 



E. Generalization of Identities and Relations to Feedback Controlled Systems 



Very recently, several important identities and nonequilibrium equalities of Statistical Physics have been generalized 
to enlarge their range of applicability to feedback systems. Sagawa and Ueda [4(1 - |49| have extended the fluctuation 
theorem and the Jarzynski relation [501 ] to feedback controlled systems even for multiple measurements. Once again, 
they show that the mutual information plays a crucial role to reformulate the nonequilibrium statistical mechanics to 
feedback controlled processes. Abreu and Seifert have extended the Hatano and Sasa equality to feedback controlled 
processes [451 ], considering systems that, even for fixed control parameters, have nonequilibrium steady states with 
dissipation. These results also provide bounds for the power output that can be extracted under isothermal condi- 
tions. The detailed fluctuation theorem has also been generalized to processes with feedback in j5lT[53| . All these 
results broadly extend the available resources to study feedback controlled systems and to find the bounds to their 
performance. 



III. OPEN PROBLEMS 



In this section we present several open problems in the context of the connections between the study of feedback 
controlled systems and information theory. We have selected those questions that we think would imply a greater 
progress in the understanding of these connections, and would further advance the understanding of feedback controlled 
systems. The questions we have selected mainly concern the establishment of bounds to the performance and the 
establishment of the thermodynamics of feedback controlled systems. Other open questions on the links between 
feedback control and information theory are raised in the Bechhoefer review on control Regarding other open 
questions that concern the implications of the theoretical results for the performance limitations of particular systems, 
due to their particular relevance we discuss here feedback controlled flashing ratchets, and also, nanotechnology devices 
and biological systems. 



7 



A. Feedback Controlled Systems: Limitations to the Performance and Thermodynamics 

Feedback controlled systems are usually designed with the aim that the controller profits from the information 
about the system state it receives in order to increase the performance of the system. Therefore, the first natural 
main question is how much the performance can be increased with a certain amount of information, or, stated 
otherwise, which are the bounds of the performance increase. The second main question is the establishment of the 
thermodynamics of feedback controlled systems accounting for thermodynamic effects of the information transfer. 
This second question can be seen as included in the first one, as, for instance, the computation of the maximum 
efficiency involves a bound in the work that can be extracted from the feedback controlled system. 

The case of a single operation of a feedback controller has been extensively studied in the context of Maxwell's 
demon and in computation theory. In this case the main thermodynamical results are known; see Sections III Al and 

EU 

On the other hand, the study of the repeated operations of a feedback controller presents new ingredients. We have 
to distinguish whether the operations of the feedback controller are correlated or uncorrelatcd. In the latter case, 
the system has lost memory of the previous actuation of the controller when its status is measured again to decide 
the next actuation by the controller. Therefore, the uncorrelated operations of the feedback can be decomposed as a 
succession of independent single operations, and thus the known previous results can be applied. However, the case 
of correlated repeated operation of the feedback controller has fundamental new ingredients that must be taken into 
account. 



1. Correlated Repeated Actuation of the Feedback Controller 

In [l]| (see Section III D|) the entropy reduction due to the repeated operation of the controller has been derived 
from the point of view of the feedback controlled system. This point of view is also used to derive the result of 0-[IH 
(see Section III Bp . From this point of view, the controller is an external agent and we are not concerned about its 
internal state. We are only concerned about the effects on the system due to its interaction with the controller. One 
of the effects is the reduction of the system's entropy due to the additional determination of the system's macrostate 
through the information obtained from the system by the controller. 

Therefore, there are still open questions to be answered. One particular open question involves the precise relations 
between these recent results and the algorithmic complexity approach adopted by Zurek [f| @] to study the correlated 
repeated actuation of a feedback controller from the perspective of the system plus controller. This study has allowed 
Zurek to establish the minimum energetic cost for erasing a memory that has stored the measurements the controller 
has processed (see Section III A|) . This minimum energetic cost also sets thermodynamic constraints that lead to a 
maximum entropy reduction for the system. 

Another open question is the application or the extension of the results found in [l3j to find bounds for other 
performance measures of feedback controlled systems, for example, the maximum attainable flux or power. 



2. Continuous Actuation of the Feedback Controller 

Continuous actuation of the feedback controller is effectively present when the controller repeatedly actuates on 
the system with a period shorter than any of the characteristic time scales of the system dynamics. It is therefore a 
limiting case of the correlated repeated actuation discussed above. However, the continuous time limit of the results 
found in [l3| is not direct because of the nontrivial relationship between differential entropy and discrete entropy 
(see [13 )• Therefore, even the extension of the results in [l3| to the continuous actuation limit remains still an open 
problem. 

Some techniques of filtering theory might be useful for the study of the information and entropy flows associated 
with continuous-time processes in non-equilibrium statistical mechanical systems [HiJ . In addition, methods of optimal 
control theory have been proven useful to compute limits on how much work can be extracted from linear systems [55| . 



B. Feedback Controlled Flashing Ratchets: Limitations to the Performance and Thermodynamics 

Feedback flashing ratchets are relevant feedback controlled systems that would provide relevant applications of the 
general results obtained for feedback controlled systems. In addition, they can provide enlightening examples to guide 
the progress in the solution of the previous open problems. 
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The application of the results of [l3j] to feedback controlled flashing ratchets has not been done yet. It will allow to 
establish bounds on the entropy reduction attainable for feedback flashing ratchet and to establish the thermodynamics 
of these systems, which are theoretically and technologically relevant. Therefore, this is another relevant open problem. 

Previously, in Section III C[ we have summarized the bounds that have been found in (38l . |39| for the performance 
increase thanks to the information used by the feedback controller, when the performance is measured by the flux (38[ 
or by the power output [33 |. These relation were stated in terms of the information per actuation. However, how to 
restate these relations in terms of the total amount of information per unit time (i.e., only computing non-redundant 
information per unit time) is still an open question. This restatement will help to put these results pa . |39| in relation 
with those found in [l3| for the entropy reduction and the thermodynamics of a general feedback controlled system. 



C. Limitations to the Operation of Nanotechnology Devices and Biological Systems 



In many nanotechnology devices and biological systems, one is interested in increasing the performance of one part 
of the system through the bidirectional interaction of other part of the system |15l - t23l |31L [33l - l35| . Therefore, we can 
think of the subsystem whose performance we want to increase as a feedback controlled system, and consider the 
other interacting part as the controller. Thus, the bounds of the performance of the feedback controlled systems will 
imply limitations to the operation of nanotechnology devices and biological systems. Finding these implications is 
still a very relevant open question. 

An illustrative example of this type of questions is the previously mentioned application of information theory to 
biochemical reaction networks to show the limits on the ability to control fluctuations in molecular abundances inside 



the cells as a function of the number of signaling events [14j . 



This shows that a deeper knowledge of the thermodynamics and the statistical physics of feedback controlled 
systems will bring a deeper understanding of nanotechnology devices and biological systems, and consequently it will 
increase our capacity of designing and controlling them. 



IV. CONCLUSIONS 



Feedback or closed-loop control is present whenever there is a controller or external agent that gathers information 
on the state of the system and uses this information to decide the action it will perform on the system. Feedback 
control allows to increase the performance, and it is present in many systems with interest for physicists, engineers 
and biologists 

We have briefly reviewed the main ideas of the historical results by Landauer Bennett Q , and Zurek [f| @] , as 
well as the more recent results on the limits to the control of a general system [l(| El , the limits to the performance of 
feedback controlled ratchets [HI, [39| , the thermodynamics of feedback controlled systems [l3j] , and the generalization of 
statistical physics identities and relations to feedback controlled systems [45M49L [5ll - [53j . This brief review has helped 
us to give the context for the open questions that we have presented in Section IIII1 These questions concern the 
quantification of the limits to the performance increase that can be obtained thanks to a feedback that uses a certain 
amount of information and the establishment of the thermodynamics of feedback controlled systems. Answering these 
questions is an important task for the theoretical understanding of feedback controlled systems, which has important 
technological applications. Feedback flashing ratchets [l5l - l23l . l3l| and Markovian particle pumps [13, 1111 are examples 
of paradigmatic motors whose deeper understanding will benefit the comprehension of the biological and technological 
devices they model, and boost new applications. Control engineers have also addressed similar topics from the point 
of view of dynamical systems [H3, [55[ using models and techniques of optimal control theory 0, [56[ as the Linear- 
Quadratic-Gaussian control or the Kalman filter. The development of the links between physics, information theory 
and control theory will allow to quantify and to establish constraints to the increase of performance reachable with a 
given amount of information in general feedback controlled systems. 
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